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Abstract 

This paper presents an algorithm to find the correspondences of points representing dominant 
feature in robot stereo vision. The algorithm consists of two main steps: dominant point extraction and 
dominant point matching. In the feature extraction phase, the algorithm utilises the widely used 
Moravec Interest Operator and two other operators: the Prewitt Operator and a new operator called 
Gradient Angle Variance Operator. The Interest Operator in the Moravec algorithm was used to 
exclude featureless areas and simple edges which are oriented in the vertical, horizontal, and two 
diagonals. It was incorrectly detecting points on edges which are not on the four main directions 
(vertical, horizontal, and two diagonals). The new algorithm uses the Prewitt operator to exclude 
feature less areas, so that the Interest Operator is applied only on the edges to exclude simple edges and 
to leave interesting points. This modification speeds-up the extraction process by approximately 5 
times. The Gradient Angle Variance (GAV), an operator which calculates the variance of the gradient 
angle in a window around the point under concern, is then applied on the interesting points to exclude 
the redundant ones and leave the actual dominant ones. The matching phase is performed after the 
extraction of the dominant points in both stereo images. The matching starts with dominant points in 
te eft image and does a local search, looking for corresponding dominant points in the right image. 
The search is geometrically constrained the epipolar line of the parallel-axes stereo geometry and the 
maximum disparity of the application environment. If one dominant point in the right image lies in the 
search areas, then it is the corresponding point of the reference dominant point in the left image A 
parameter provided by the GAV is thresholded and used as a rough similarity measure to select the 
corresponding dominant point if there is more than one point the search area. The correlation is used as 
a final decision tool when there is still more than one point in the search area. If there is no dominant 
point in the search area or if the points in the search area are below a correlation threshold, than the 
dominant point in the reference image is occluded and can not be corresponded. The algorithm has been 
modelled, implemented and shown to be fast, robust and parallel. The parallelism is created from three 
mam features: locality of the operators; a memory optimisation scheme; and the ability to fully 
parallelise the extraction phase which is the most computational intensive task in the algorithm. The last 
feature is achieved by performing the extraction phase on the two images simultaneously. 
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Introduction: 


Range-finding techniques in robotics are the techniques used to find the range, range 
map, or the 3-D structure of an object in the workspace. They can be classified as either 
passive or active [JAR 83] Active techniques require artificial sources to illuminate the 
workspace, whereas passive techniques do not need such sources. Typically passive 
techniques require simpler and cheaper set-up than active methods. 

Stereopsis, which is the utilisation of multi-views (images from different 
prospectives) for finding the range, is a very efficient passive technique. In stereopsis, 
correspondence is established between features that are projection of the same physical 
identity in each view. Then a geometric transformation called triangulation, which uses 
the corresponding points and the imaging geometry, is used to calculate the range, the 
range map, or the 3-D structure of the scene. 

The major issues in characterising existing stereopsis algorithms are [DHO 89], 
[HAR 93]: imaging geometry, matching primitive and matching measure. Imaging 
geometry is characterised by the mutual orientation of the cameras (either parallel [TU 
90] [KIM 87] or non parallel [AYA 87] ), and the number of cameras used (either two 
"binocular" [LLO 87], three "trinocular" [ITO 86], or more). There are different 
matching primitive such as: most variance points [HAN 74], interesting points [MOR 
80], high curvature points [DER 90], edges [BAK 82], intensity values [BAR 86], 
straight lines [LIU 91] The matching measure is either: correlation [HAN 74] [FRY 89], 
intensity difference [OHT 85], or any characteristic of the matching primitive such as: 
sign strength [BAK 82], direction sign [GRI 85], and orientation sign [AYA 87], Some 
of the stereopsis algorithms use matching optimisation schemes. The main matching 
optimisation schemes are: multistage matching [SHI 87], nearest neighbour[GRI 85], 
relaxation [BAR 80], and dynamic programming [OHT 85]. 

The main problem in stereopsis is that it is computational intensive, which makes 
parallel processing one of the best solutions [DRU 86], In this paper, a parallel algorithm 
for corresponding dominant features is presented. The algorithm utilises conventional 
parallel axis binocular stereo geometry. It uses dominant points which are modified 
interesting points, as a matching primitive. The matching measure used in the algorithm 
is correlation. The algorithm consists of two main phases: dominant feature extraction 
and matching. The algorithm is found to give satisfactory results. 


Dominant Feature Extraction: 

In this phase the algorithm utilises the widely used Moravec Interest Operator [MOR 
80], [BAR 80], [JEN 86], [LEE 90] and two other operators: Prewitt Operator [PRE 
70] and a new operator called Gradient Angle Variance (GAV). Moravec Operator for a 
certain point is the sum of squares of differences of adjacent pixels in each of the four 
directions ( horizontal, vertical, and two diagonals) over a small window. Moravec 
interesting points are those which has the interest operator as a local maxima [MOR 80], 
Moravec Interest Operator is able to exclude featureless areas and simple edges 
which are oriented in the vertical, horizontal, and diagonals directions. The 



computational performance of the operator op can be approximately formulated in 
terms of the number arithmetic operations, multiplications and additions . 

°Pm 0 r * 4 [( m ' 2 - w >)mul + (w : - w)sub+(w 1 -w - 1 )add] 

where w is Moravec window size. And since addition and subtraction are considered to 
take the same time: 

°Pmor * 4 [(^ 2 - *)mul + (2w 2 - 2 w - 1 )add] 


In the proposed algorithm, Prewitt operator is performed first to exclude featureless 
areas. The gradients of Prewitt operator are: 

G x = (*7 + *« +*»)-(*, +z 2 + z } ) 

G y ~ 2 6 Z 9 ) ~ (*| + + “7 ) 

g=|g,|+|g,| 

The computational performance of the Prewitt operator with 3 as the mask size, is 
in terms of the number of arithmetic operations is op pn : 

°P P n~^ °dd + 2 sub + 2 mod 
op pr , ~ 1 1 add + 2mul 

When both operators are combined, so that the Prewitt is performed first, the 
combined operator Prewitt-Moravec has op pmo operations as: 

°Ppn, 0 -OP prt +p*0 Pmor 

where p is the edges density which depends on the features density in the image. For an 
image with p =1/8 and 3^=5 the number of operations is : 

°Ppr. =* 2 mul + 1 1 add 

°Pmor = 80 / 77 /// +\S6add 

°P pmo = 1 2 / 7 / ul + 3 1 add 

comparing op pmo and op mor , it give a speed-up factor of approximately 6. 

The improvement discussed so far is speed wise. To overcome detecting redundant 
points, a new operator called Gradient Angle Variance has been introduced. GAV of a 
point is the variance in the gradient angle of pixels which lies in a thin (one pixel wide) 
edge in a window around the point under concern. The thin edge points are detected 
with an operator which emphasises points with local amxima gradient [SHI 87], GAV is 
applied to the points detected by Prewitt-Moravec, so that a very low GAV means 
redundant point. The GAV is shown to effect the speed of the extraction by around 10% 
which mans that the overall speed-up factor of the extraction is approximately 5. 
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Matching: 


After dominant features are detected in both images , the matching process is 
applied for establishing correspondence between the reference dominant features in the 
left image and the corresponding dominant features in the right image. The main issues in 
the matching process are, the search area and the matching measure. The search area is a 
bounded area in the right image at which search for correspondent of a certain reference 
dominant feature is taking place, whereas correlation is the matching measure. The 
matching process has been speeded-up by reducing the search space and minimising the 
need to the correlation which is a reliable but computational intensive matching measure. 

The search area has been reduced using Parallel-axis (Conventional) stereo imaging 
geometry as in Fig. 1 [BAR 82], [KIM 87], [TU 90], This configuration constrained the 
search area vertically and horizontally. In the vertical direction the constraints are the 
epipolar line with a little tolerance of few pixels up and down. This tolerance is due to 
the practical consideration of aligning the vocal axis of the two cameras. In the 
horizontal direction, the search area would be on the left side of the transferred 
coordinates of the reference dominant feature and within the maximum disparity 
[KIM 87] 



where is the maximum disparity, and D ^ is the minimum possible distance for 

the undertaken application [KEM 87], This boundary of the search area would make it 
part of the right image which is containing a subset; nil, one or more ; of the dominant 
features in the right image. The dominant features in this subset are candidates to be 
correspondent in one of the following three cases. The first is, when there are no features 
in the search area which means that the reference dominant feature is occluded and 
cannot be corresponded. The second is, when there are only one feature, which means 
that it is the correspondent of the reference dominant feature. The third case is, when 
there is more than one dominant feature in the search area, a matching measure is needed 
to decide the correspondent if it exists. 

The correlation is the matching measure to decide the correspondence. To minimise 
the need to the correlation, a parameter provided by the GAV operator called NGAV is 
used as rough similarity measure to exclude some of the candidate features which are not 
similar to the reference feature. The Number of pixels in the GAV window ( NGAV) is 
used to rule out those candidats which are not satisfing the following condition: 

\NGA V [r] - NGA V [c, ]| < thp* NGA V [r] 

where NGAV[r] and NGAV[c] are the NGAV of the reference dominant feature and the 
NGAV of the candidate dominant feature respectivly, and thp is the thresholding 
percentage(e.g. 0.10, 0.20, or 0.30). 
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The result of applying NGAV has three cases, similar to those of the search area. If it 
is the third case, in which more than one candidate are satisfying the above thresholding 
condition, then it is the role of the correlation to select best match among the candidates 
which are above certain threshold of the correlation coefficient (e g. 0.7 or 0.8). If none 
of the candidates is above that threshold, this means that the reference feature is 
occluded. 


Parallelism: 

Parallelism is a major step for Real-Time realisation of stereo algorithms. The 
algorithm has a high potential parallelism which is created by different features. The first 
is the locality of the operators (Moravec, Prewitt, and GAV) used in the feature 
extraction, which means that an operator is performed on a mask bases. This feature 
enables more than one or even many processors to perform the operator to image 
portions on the same time with a very little overhead and reduces interprocessor 
communication to minimum. The second is that the algorithm optimises the memory 
space in calculating local maxima by avoiding memory space needed to keep the result of 
Moravec operator for the comparison with the neighbouring pixels. The third is the 
ability to fully parallelise the extraction phase which is the most computational intensive 
task in the algorithm. The last feature is achieved by performing the extraction phase on 
the two images simultaneously. 


Results and Discussion: 

The algorithm (Fig. 2) has been implemented and shown to detect and correspond 
nearly all dominant features, with speeding-up the feature extraction phase by a factor of 
5 which is very close to the one obtained theoretically. Fig. 3 shows the features 
extracted by Moravec and even by Prewitt-Moravec is the same, whereas Fig. 4 show 
how GAV excluded the redundant features. Fig. 5a shows the reference dominant 
features in the left image, whereas Fig. 5b shows the corresponding dominant features in 
the right image. The output has been produced on binary thinned images for clarity 
reasons. The disparity of certain corresponding pair of points is the distance between 
them after transferring one point into the other coordinate system as in Fig 1 . 

Although the algorithm shows a reasonable amount of robustness, mismatching is 
not impossible. Mismatching could happen due to, misdetection of some dominant 
features, repeated patterns in the scene, or occlusion. Misdetection can be solved by 
increasing the directions in which variance is calculated, and by increasing the size of the 
Moravec window. Mismatching due to repeated pattern and occlusion can be detected 
by a scheme called local disparity voting. In this scheme the disparity of each point 
compared locally with the disparity of neighbouring points, if the disparity value is 
different from the values of the majority it means mismatching. Once mismatching is 
detected, the corresponding feature is excluded and other candidates in the search area 
are rematched to the reference feature. 
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Conclusion: 


In this paper an algorithm for corresponding dominant features in robot stereo vision 
has been presented. The algorithm is composed of two main parts, dominant feature 
extraction and matching. In the extraction phase the algorithm utilises three operators 
Prewitt, Moravec and GAV. Prewitt is used for detecting features, Moravec is to detect 
interesting points which are candidates to be dominant, and GAV is for detecting and 
excluding redundant points. In matching, correlation has been used as the matching 
measure. The matching time has been reduced using parallel axis stereo geometry and the 
rough similarity measure NGAV. 

The algorithm has been modelled, implemented and shown to be fast, robust and 
parallel. Although the robustness that the algorithm is shown to have, mismatching is not 
impossible and some solution has been proposed to minimise it. 
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Fig. 1 Conventional Parallel Axis Stereo Geometry [KIM 87] 
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Fig. 2 Flowchart of the algorithm 
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Fig. 3 The result of applying MORAVEC Operator 




Fig. 4 The result of the combining PREWITT, MORAVEC and GAV Operators 




(b) 


^8' ^ The result of corresponding dominant features 
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